feat(socratic-recovery): derive quality scenarios instead of blanket [OPEN]#499
Merged
rdmueller merged 1 commit intoMay 17, 2026
Conversation
…[OPEN] Closes LLM-Coding#498. A recovery run reduced arc42 Chapter 10 (Quality Requirements) to two [OPEN] pointers, which an independent arc42 review graded RED. The Socratic process was treating all of Quality as team-only knowledge. It is not. Quality-goal *priority/ranking* is team knowledge (Q4.9, [OPEN]); quality *scenarios* — what the system measurably does today — are usually code-derivable. Made the split operative (it previously lived only as a "rule of thumb" in references/iso-25010.md, with nothing enforcing it): - contract (LLM-Coding#493) + phase-1 prompt: Q4 quality scenarios are [ANSWERED] where code shows measurable behaviour (timeouts, truncation limits, budgets, retry policies, the Q3.8 threats and test concept); only the ranking is [OPEN]; never invent a target number. - iso-25010.md: new section distinguishing a literal-threshold scenario (fully [ANSWERED], cite file:line) from a mechanism-only one (scenario derived, target value [OPEN]); explicit Q3.8 evidence reuse. - arc42.md: Q3.10 row now defines the Q4 -> Chapter 10 relationship. - phase-2-synthesize.md: Chapter 10 carve-out — synthesized from the [ANSWERED] Q4 scenarios plus the Q4.9 ranking, never just an [OPEN] pointer; "No information" only legitimate if Q4 produced no answered scenario at all. - examples.md: added an [ANSWERED] quality-scenario leaf (a literal 3-second timeout) alongside the existing mechanism/[OPEN]-target pair. Also fixed the German contract's stop-rule wording to match the EN one from the LLM-Coding#497 CodeRabbit pass (was still "Stakeholder-Fakt"). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Contributor
WalkthroughDie Pull Request präzisiert die Socratic Code-Theory Recovery Methodik zur Behandlung von Qualitätszielen (Q4): Messbare Qualitätsszenarien aus dem Code werden als ÄnderungenQ4 Qualitätsszenarien und arc42 Kapitel 10 Synthese
🎯 2 (Simple) | ⏱️ ~12 Minuten Möglicherweise relevante Pull Requests
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In
`@plugins/semantic-anchors/skills/socratic-code-theory-recovery/references/examples.md`:
- Around line 167-176: The fenced code block that begins with "[ANSWERED]" in
the examples.md snippet is missing a language tag (causing MD040); update the
opening fence from ``` to ```text (and ensure the closing fence remains ```), so
the block becomes a ```text fenced code block containing the existing lines
starting with "[ANSWERED]" through "arc42 Chapter 10."
In `@skill/socratic-code-theory-recovery/references/examples.md`:
- Around line 167-176: The Markdown fenced code block containing the
"[ANSWERED]" evidence/performance scenario is missing a language specifier which
triggers MD040; fix it by adding a language tag (e.g., "text" or "md") after the
opening triple backticks for the fenced block that starts with "[ANSWERED]" so
the linter recognizes the fence, leaving the block contents unchanged and
keeping the closing triple backticks intact.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yml
Review profile: CHILL
Plan: Pro
Run ID: 0362cd41-9852-4de1-a116-e0cf386b5133
📒 Files selected for processing (14)
docs/brownfield-workflow.adocdocs/brownfield-workflow.de.adocplugins/semantic-anchors/skills/socratic-code-theory-recovery/prompts/phase-1-question-tree.mdplugins/semantic-anchors/skills/socratic-code-theory-recovery/prompts/phase-2-synthesize.mdplugins/semantic-anchors/skills/socratic-code-theory-recovery/references/arc42.mdplugins/semantic-anchors/skills/socratic-code-theory-recovery/references/examples.mdplugins/semantic-anchors/skills/socratic-code-theory-recovery/references/iso-25010.mdskill/socratic-code-theory-recovery/prompts/phase-1-question-tree.mdskill/socratic-code-theory-recovery/prompts/phase-2-synthesize.mdskill/socratic-code-theory-recovery/references/arc42.mdskill/socratic-code-theory-recovery/references/examples.mdskill/socratic-code-theory-recovery/references/iso-25010.mdwebsite/public/data/contracts.jsonwebsite/public/llms.txt
Comment on lines
+167
to
+176
| ``` | ||
| [ANSWERED] | ||
| Evidence: src/integration/InventoryClient.java:34 (Duration.ofSeconds(3)), src/config/HttpClientConfig.java:19 | ||
| Performance scenario: when OrderService reserves inventory, the call to | ||
| the Inventory service is bounded by a 3-second timeout; on timeout the | ||
| order fails with 409 Conflict rather than hanging. The 3-second value is | ||
| a literal in the code — a measurable, code-derived target. This is an | ||
| [ANSWERED] quality scenario, not an [OPEN] one: the threshold did not | ||
| have to be invented or asked of the team. It belongs in arc42 Chapter 10. | ||
| ``` |
Contributor
There was a problem hiding this comment.
Sprachkennung im Fence ergänzen
In Line 167 startet ein Codeblock ohne Sprachkennung; das triggert MD040. Bitte z. B. text setzen, damit Linting stabil bleibt.
Vorgeschlagener Fix
-```
+```text
[ANSWERED]
Evidence: src/integration/InventoryClient.java:34 (Duration.ofSeconds(3)), src/config/HttpClientConfig.java:19
Performance scenario: when OrderService reserves inventory, the call to
@@
[ANSWERED] quality scenario, not an [OPEN] one: the threshold did not
have to be invented or asked of the team. It belongs in arc42 Chapter 10.
-```
+```🧰 Tools
🪛 markdownlint-cli2 (0.22.1)
[warning] 167-167: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In
`@plugins/semantic-anchors/skills/socratic-code-theory-recovery/references/examples.md`
around lines 167 - 176, The fenced code block that begins with "[ANSWERED]" in
the examples.md snippet is missing a language tag (causing MD040); update the
opening fence from ``` to ```text (and ensure the closing fence remains ```), so
the block becomes a ```text fenced code block containing the existing lines
starting with "[ANSWERED]" through "arc42 Chapter 10."
Comment on lines
+167
to
+176
| ``` | ||
| [ANSWERED] | ||
| Evidence: src/integration/InventoryClient.java:34 (Duration.ofSeconds(3)), src/config/HttpClientConfig.java:19 | ||
| Performance scenario: when OrderService reserves inventory, the call to | ||
| the Inventory service is bounded by a 3-second timeout; on timeout the | ||
| order fails with 409 Conflict rather than hanging. The 3-second value is | ||
| a literal in the code — a measurable, code-derived target. This is an | ||
| [ANSWERED] quality scenario, not an [OPEN] one: the threshold did not | ||
| have to be invented or asked of the team. It belongs in arc42 Chapter 10. | ||
| ``` |
Contributor
There was a problem hiding this comment.
Code-Fence in Markdown mit Sprache deklarieren
In Line 167 fehlt die Sprachangabe am Fence (```). Das verursacht den MD040-Hinweis und ist schnell behebbar.
Vorgeschlagener Fix
-```
+```text
[ANSWERED]
Evidence: src/integration/InventoryClient.java:34 (Duration.ofSeconds(3)), src/config/HttpClientConfig.java:19
Performance scenario: when OrderService reserves inventory, the call to
@@
[ANSWERED] quality scenario, not an [OPEN] one: the threshold did not
have to be invented or asked of the team. It belongs in arc42 Chapter 10.
-```
+```Based on learnings, in skill/ sollen Inhalte konsistent als Markdown gepflegt und entsprechend sauber gelintet werden.
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| ``` | |
| [ANSWERED] | |
| Evidence: src/integration/InventoryClient.java:34 (Duration.ofSeconds(3)), src/config/HttpClientConfig.java:19 | |
| Performance scenario: when OrderService reserves inventory, the call to | |
| the Inventory service is bounded by a 3-second timeout; on timeout the | |
| order fails with 409 Conflict rather than hanging. The 3-second value is | |
| a literal in the code — a measurable, code-derived target. This is an | |
| [ANSWERED] quality scenario, not an [OPEN] one: the threshold did not | |
| have to be invented or asked of the team. It belongs in arc42 Chapter 10. | |
| ``` |
🧰 Tools
🪛 markdownlint-cli2 (0.22.1)
[warning] 167-167: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@skill/socratic-code-theory-recovery/references/examples.md` around lines 167
- 176, The Markdown fenced code block containing the "[ANSWERED]"
evidence/performance scenario is missing a language specifier which triggers
MD040; fix it by adding a language tag (e.g., "text" or "md") after the opening
triple backticks for the fenced block that starts with "[ANSWERED]" so the
linter recognizes the fence, leaving the block contents unchanged and keeping
the closing triple backticks intact.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #498.
Problem
A recovery run reduced arc42 Chapter 10 (Quality Requirements) to two
[OPEN]pointers, which an independent arc42 review graded RED. The Socratic process treated all of Quality as team-only knowledge.It is not: quality-goal priority/ranking is team knowledge (
Q4.9,[OPEN]); quality scenarios — what the system measurably does today — are usually code-derivable.Fix
The mechanism/target split already existed as a "rule of thumb" in
references/iso-25010.md, but nothing operative enforced it. Made it operative:[ANSWERED]where code shows measurable behaviour (timeouts, truncation limits, budgets, retry policies, theQ3.8threats and test concept); only the ranking is[OPEN]; never invent a target number.iso-25010.md— new section distinguishing a literal-threshold scenario (fully[ANSWERED], citefile:line) from a mechanism-only one (scenario derived, target value[OPEN]); explicitQ3.8evidence reuse.arc42.md— theQ3.10row now defines theQ4→ Chapter 10 relationship.phase-2-synthesize.md— Chapter 10 carve-out: synthesized from the[ANSWERED]Q4 scenarios plus theQ4.9ranking, never just an[OPEN]pointer.examples.md— added an[ANSWERED]quality-scenario leaf (a literal 3-second timeout) next to the existing mechanism/[OPEN]-target pair.The anti-fabrication guard is explicit throughout: "derive scenarios" means reading measurable behaviour from code, never guessing plausible numbers.
Also fixed the German contract's stop-rule wording to match the EN one from the #497 CodeRabbit pass (was still "Stakeholder-Fakt").
Test plan
npm run buildsucceeds;/contracts,/brownfieldrenderllms.txtshows the updated socratic-code-theory-recovery contract[OPEN]pointer🤖 Generated with Claude Code
Summary by CodeRabbit